Relevance- and interface-driven clustering for visual information retrieval

Search results of spatio-temporal data are often displayed on a map, but when the number of matching search results is large, it can be time-consuming to individually examine all results, even when using methods such as filtered search to narrow the content focus. This suggests the need to aggregate results via a clustering method. However, standard unsupervised clustering algorithms like $K$-means (i) ignore relevance scores that can help with the extraction of highly relevant clusters, and (ii) do not necessarily optimize search results for purposes of visual presentation. In this article, we address both deficiencies by framing the clustering problem for search-driven user interfaces in a novel optimization framework that (i) aims to maximize the relevance of aggregated content according to cluster-based extensions of standard information retrieval metrics and (ii) defines clusters via constraints that naturally reflect interface-driven desiderata of spatial, temporal, and keyword coherence that do not require complex ad-hoc distance metric specifications as in $K$-means. After comparatively benchmarking algorithmic variants of our proposed approach – RadiCAL – in offline experiments, we undertake a user study with 24 subjects to evaluate whether RadiCAL improves human performance on visual search tasks in comparison to $K$-means clustering and a filtered search baseline. Our results show that (a) our binary partitioning search (BPS) variant of RadiCAL is fast, near-optimal, and extracts higher-relevance clusters than $K$-means, and (b) clusters optimized via RadiCAL result in faster search task completion with higher accuracy while requiring a minimum workload leading to high effectiveness, efficiency, and user satisfaction among alternatives.     

Modelling of chromatic contrast for retrieval of wallpaper images

Colour remains one of the key factors in presenting an object and, consequently, has been widely applied in retrieval of images based on their visual contents. However, a colour appearance changes with the change of viewing surroundings, the phenomenon that has not been paid attention yet while performing colour‐based image retrieval. To comprehend this effect, in this article, a chromatic contrast model, CAMcc, is developed for the application of retrieval of colour intensive images, cementing the gap that most of existing colour models lack to fill by taking simultaneous colour contrast into account. Subsequently, the model is applied to the retrieval task on a collection of museum wallpapers of colour‐rich images. In comparison with current popular colour models including CIECAM02, HSI and RGB, with respect to both foreground and background colours, CAMcc appears to outperform the others with retrieved results being closer to query images. In addition, CAMcc focuses more on foreground colours, especially by maintaining the balance between both foreground and background colours, while the rest of existing models take on dominant colours that are perceived the most, usually background tones. Significantly, the contribution of the investigation lies in not only the improvement of the accuracy of colour‐based image retrieval but also the development of colour contrast model that warrants an important place in colour and computer vision theory, leading to deciphering the insight of this age‐old topic of chromatic contrast in colour science. © 2014 Wiley Periodicals, Inc. Col Res Appl, 40, 361–373, 2015     

ZerNet: Convolutional Neural Networks on Arbitrary Surfaces Via Zernike Local Tangent Space Estimation

In this paper, we propose a novel formulation extending convolutional neural networks (CNN) to arbitrary two-dimensional manifolds using orthogonal basis functions called Zernike polynomials. In many areas, geometric features play a key role in understanding scientific trends and phenomena, where accurate numerical quantification of geometric features is critical. Recently, CNNs have demonstrated a substantial improvement in extracting and codifying geometric features. However, the progress is mostly centred around computer vision and its applications where an inherent grid-like data representation is naturally present. In contrast, many geometry processing problems deal with curved surfaces and the application of CNNs is not trivial due to the lack of canonical grid-like representation, the absence of globally consistent orientation and the incompatible local discretizations. In this paper, we show that the Zernike polynomials allow rigourous yet practical mathematical generalization of CNNs to arbitrary surfaces. We prove that the convolution of two functions can be represented as a simple dot product between Zernike coefficients and the rotation of a convolution kernel is essentially a set of 2 × 2 rotation matrices applied to the coefficients. The key contribution of this work is in such a computationally efficient but rigorous generalization of the major CNN building blocks.     

Robust image retrieval by cascading a deep quality assessment network

The performance of computer vision algorithms can severely degrade in the presence of a variety of distortions. While image enhancement algorithms have evolved to optimize image quality as measured according to human visual perception, their relevance in maximizing the success of computer vision algorithms operating on the enhanced image has been much less investigated. We consider the problem of image enhancement to combat Gaussian noise and low resolution with respect to the specific application of image retrieval from a dataset. We define the notion of image quality as determined by the success of image retrieval and design a deep convolutional neural network (CNN) to predict this quality. This network is then cascaded with a deep CNN designed for image denoising or super resolution, allowing for optimization of the enhancement CNN to maximize retrieval performance. This framework allows us to couple enhancement to the retrieval problem. We also consider the problem of adapting image features for robust retrieval performance in the presence of distortions. We show through experiments on distorted images of the Oxford and Paris buildings datasets that our algorithms yield improved mean average precision when compared to using enhancement methods that are oblivious to the task of image retrieval. ¹     

Efficient Identification of water conveyance tunnels siltation based on ensemble deep learning

The inspection of water conveyance tunnels plays an important role in water diversion projects. Siltation is an essential factor threatening the safety of water conveyance tunnels. Accurate and efficient identification of such siltation can reduce risks and enhance safety and reliability of these projects. The remotely operated vehicle (ROV) can detect such siltation. However, it needs to improve its intelligent recognition of image data it obtains. This paper introduces the idea of ensemble deep learning. Based on the VGG16 network, a compact convolutional neural network (CNN) is designed as a primary learner, called Silt-net, which is used to identify the siltation images. At the same time, the fully-connected network is applied as the meta-learner, and stacking ensemble learning is combined with the outputs of the primary classifiers to obtain satisfactory classification results. Finally, several evaluation metrics are used to measure the performance of the proposed method. The experimental results on the siltation dataset show that the classification accuracy of the proposed method reaches 97.2%, which is far better than the accuracy of other classifiers. Furthermore, the proposed method can weigh the accuracy and model complexity on a platform with limited computing resources.     

基于Flink的海量医学图像检索系统设计与实现

医学图像检索是有效利用医学资源的基础，而医学图像的海量性和增量性为图像检索带来了新的挑战和要求。为了提高医学图像检索过程的效率，设计并实现一种基于Flink的海量医学图像检索系统。首先，系统通过Web应用作为用户操作入口，在后端搭建数据平台和业务集群；其次，系统通过HBase对医学图像数据进行分布式存储，利用深度卷积神经网络模型提取医学图像特征；然后，将所提取的医学图像特征数据进行乘积量化编码，并通过HBase进行存储；最后，通过基于Flink的内存计算对接Kafka进行实时图像检索，以及对批量导入图像的特征索引编码。系统在4个节点的服务器上部署分布式集群，使用真实医学图像数据集进行测试，通过在MapReduce和Spark两种不同技术模块下的对比实验表明本系统具有更好的检索效率表现。     

A tutorial on the segmentation of metallographic images: Taxonomy,new MetalDAM dataset,deep learning-based ensemble model,experimental analysis and challenges

Image segmentation is an important issue in many industrial processes, with high potential to enhance the manufacturing process derived from raw material imaging. For example, metal phases contained in microstructures yield information on the physical properties of the steel. Existing prior literature has been devoted to develop specific computer vision techniques able to tackle a single problem involving a particular type of metallographic image. However, the field lacks a comprehensive tutorial on the different types of techniques, methodologies, their generalizations and the algorithms that can be applied in each scenario. This paper aims to fill this gap. First, the typologies of computer vision techniques to perform the segmentation of metallographic images are reviewed and categorized in a taxonomy. Second, the potential utilization of pixel similarity is discussed by introducing novel deep learning-based ensemble techniques that exploit this information. Third, a thorough comparison of the reviewed techniques is carried out in two openly available real-world datasets, one of them being a newly published dataset directly provided by ArcelorMittal, which opens up the discussion on the strengths and weaknesses of each technique and the appropriate application framework for each one. Finally, the open challenges in the topic are discussed, aiming to provide guidance in future research to cover the existing gaps.